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Research into human decision making (DM) processes from outside of education paint a 
different picture of DM than current DM models in education. This pilot study assesses the 
use of critical decision method (CDM) - developed from observations of firefighters’ DM - 
in the context of primary mathematics teachers’ in-class DM. Preliminary results show that 
CDM yields significant amounts of data regarding teachers’ cognition during DM and that 
the process that expert teachers follow when they make decisions may better match 
naturalistic accounts of DM. 

Research into teachers’ in-class decision making (DM) falls within a wider area of 
research regarding how humans make decisions. This study applies the lens of naturalistic 
decision making (NDM) to teacher DM. NDM research, applied widely in areas such as 
medicine, firefighting and the military has not been applied to teacher DM. In order to 
demonstrate how an NDM framework affects teacher DM research, Schoenfeld’s (2011) 
model of mathematics teachers’ DM is used to provide a counterpoint to NDM in the 
following section. Critical decision method (CDM) is a form of semi-structured interview 
designed to illuminate a decision maker’s cognition during real-time DM. While CDM has 
been applied to DM in situations of high-stakes, life-and-death DM, it may be the case that 
the DM of teachers - rarely as high stakes as the DM of a squad commander on a 
battlefield - may not be able to be studied using this method. Hence the current paper 
reports on the preliminary results of a pilot study that employs CDM to study the real-time, 
classroom-based DM of expert primary mathematics teachers. 

Three research questions guide this pilot study: 

1 . Can CDM be applied to the context of primary mathematics teaching? 

2. What kind of information does CDM generate when applied to this context? 

3. Does the information generated have implications for current models of teachers’ 
DM in primary mathematics classrooms? 

Literature Review 
Models of Human Decision Making 

Research regarding how humans make decisions has moved through three key phases 
(Tolcott, 1992). Rational models of DM focused on developing mathematical models 
designed to deliver optimal decisions. The calculations used to identify optimal courses of 
action were complex and these models did not seem to capture how actual human beings 
make decisions. The second phase began in the 1970s as psychologists started to present 
people with decision scenarios in laboratory environments in order to study how humans 
actually make decisions. This led to the development of normative models of DM - 
normative in that they tested humans’ DM under normal or idealised conditions. In the 
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mid-1980s, NDM models (the third phase) were developed. Rather than bringing people 
into labs, NDM research began with the premise that some people are very good at making 
decisions already, so researchers should investigate how these experts work (Lipshitz, 
Klein, Orasanu, & Salas, 2001). By investigating a range of skilled professionals making 
decisions in the field, NDM researchers have developed various models of DM which 
claim to describe humans’ natural DM processes. 

In order to illustrate how a NDM frame changes the nature of teacher DM research, 
Schoenfeld’s (2011) recent model of teacher DM is discussed. In Schoenfeld’s model, DM 
is driven by teachers’ goals. When DM is viewed through this lens, the key questions to be 
asked when analysing teacher DM are: what are the teacher’s goals; and how did those 
goals become a priority? Schoenfeld claimed that, by asking these questions, he can model 
a teacher’s DM and predict the decisions that an individual teacher will make in a given 
situation. Schoenfeld employs the idea of subjective valuations and assigns costs and 
benefits to the range of options which face a decision maker. This means that “if you know 
which options someone perceives as being available and how he or she assesses the costs, 
benefits and probabilities of the possible outcomes, you can compute each outcome’s 
subjective expected outcome” (p. 18). Teacher goals impact on how those costs, benefits 
and probabilities are assessed. Schoenfeld also argued that in some everyday situations, “it 
would be crazy to suggest that people actually engage in such computations [of subjective 
expected outcomes] during decision making” (p. 18) because of the speed of the process of 
making a decision. Thus the mechanisms by which teacher DM is assessed by Schoenfeld 
serve more to provide a means to model and predict a teacher’s DM, rather than to 
investigate the actions or processes that the teacher is engaged in during in-the-moment 
DM. While Schoenfeld claimed that this kind of analysis can explain the behaviour of the 
teacher decision maker - such as explaining why non-routine decisions were made in 
videoed lessons - it doesn’t explain the actual processes engaged in by the teacher who was 
not making the kind of cost/benefit calculations carried out in Schoenfeld’s analyses. While 
Schoenfeld acknowledges that calculations of costs/benefits cannot occur in real-time DM 
situations, his model suggests that teachers are weighing up options when they make 
decisions. This process is referred to as deliberation by DM researchers (Lipshitz et al., 
2001). Thus, according to Schoenfeld, teachers’ goals change as a lesson progresses. They 
consider multiple options to achieve those goals and weigh up their options. 

Klein’s (1997) recognition-primed decision (RPD) model is a leading NDM model. It 
was developed, primarily, by assessing the DM of experienced firefighters but has been 
applied to a wide range of fields. After analysing the cognitive processes of expert decision 
makers, the RPD model argues that concurrent deliberation of options rarely happens for 
the expert. Instead, when faced with a difficult decision, experts focus on reading the 
situation. Once they have a reading of the situation, they start to generate alternative 
courses of action one at a time. Each alternative course of action is assessed via mental 
simulation of its impact until a workable course of action is found. Expert decision makers 
tend to make better decisions because they are better at reading the situation; they tend to 
generate courses of action which will work as the first option they consider and they often 
do not deliberate concurrently between options. Novices, on the other hand, do deliberate 
concurrently (Randel, Pugh, & Reed, 1996); this has been associated with poor 
performance in decision tasks. 

According to Klein (1997), getting an accurate read of the situation is vital for expert 
decision makers as the situation suggests the courses of action which are available. For a 
seasoned firefighter, smoke billowing from one part of a burning building suggests a 
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different course of action to smoke billowing from another part of the building (Klein, 
1993). Hence, recognition of key environmental cues, facilitates the development of 
meaningful assessments of the situation, which ensures that the first options considered by 
the expert are ‘reasonably good’ ones. 

Contrasting Schoenfeld’s and Klein’s DM models illustrates two substantially different 
approaches to researching teachers’ in-class DM. According to Lipshitz et al. (2001), who 
are, admittedly, firmly in the NDM camp, Schoenfeld’s model could be characterised as 
being ‘input-output oriented’. It tries to establish the output (decision made) given certain 
inputs (teacher goals and resources). Klein’s model is ‘process oriented’ in that it seeks to 
uncover the processes which underpin effective DM. The process of DM is characterised 
by concurrent deliberation of options in Schoenfeld’s model, while Klein argues that 
experts consider options serially most of the time. While Schoenfeld developed a model of 
DM first and then sought to test whether it could be used to predict teachers’ DM, an NDM 
approach entails finding teachers who are good at making real-time decisions and trying to 
find out what they do and how they do it in ill-structured, dynamic classroom 
environments. 

Critical Decision Method ( CDM) 

CDM is a form of cognitive task analysis which has been used successfully by NDM 
researchers in a range of fields. It consists of a semi- structured interview which is 
described in more detail in the method section. CDM involves asking experts to recall 
critical incidents - incidents where their DM abilities were tested. The goal of CDM is to 
use the critical incident as a means of getting an expert decision maker to reflect on their 
cognition during a difficult decision (Crandall, Klein, & Hoffman, 2006). This generates 
data regarding the interviewee’s cognition which can be used to build a picture of the 
process of expert DM and uncover the knowledge and skills which support that process 
(Cannon-Bowers & Bell, 1997). 

Many of the situations in which CDM has been used successfully have been 
memorable, life-and-death situations. Perhaps teachers’ decisions, designed to enhance 
students’ mathematical thinking, will not be as memorable as the kind of decisions studied 
using CDM in the past and interviewees will be unable to recall critical incidents. Hence 
this pilot seeks to test the viability of this method in the context of primary mathematics 
teaching. Crandall et al. (2006) stated that CDM may not be appropriate when trying to 
capture DM in combat-like situations. While primary maths classrooms should not be 
combat situations, perhaps there could be some similarities between combat situations and 
some classrooms - they could both be situations “where people work under severely 
stressful conditions and handle very high workloads, [which] can create a blur of events 
that are difficult to recall as discrete cases” (p.84). This could lead to a lack of detail in 
teachers’ recall of a critical incident, or recall of an individual critical incident may be 
blurred with events from other incidents. It is hypothesised that those who are expert 
decision makers in primary maths classrooms will recall critical incidents in some detail. 
This pilot tests the hypothesis and also tries to ascertain the level of detail recalled. 

This study defines an incident as critical in the context of a primary maths classroom if: 

1 . the interviewed teacher identifies the incident as non-routine; 

2. the incident relates to students’ understanding of mathematics (rather than a 
behaviour management issue, for example); 

3. the interviewed teacher felt that the incident was challenging; 

4. the teacher did not have a ready response to the incident. 
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Routine incidents may be met by experts with a certain level of cognitive automaticity 
(Crandall et al., 2006). Cognition which is automatic may lead an expert to not be 
completely conscious of all aspects of the cognitive processes employed during DM. Thus, 
CDM focuses on incidents which were non-routine and challenging in the eyes of the 
interviewee. In this kind of situation, interviewees will have relied less on automatic 
responses and will be better able to recall what they were actually thinking during the 
incident. Investigation of routine incidents can lead interviewees to recall standard 
rationalised accounts of their thinking (Crandall et al., 2006). By focusing on an incident 
where the teacher did not have a ready response, it is hoped that teachers’ cognition in 
situations where they had to ‘think on their feet’ can be captured. 

Methodology 

CDM involves conducting a semi-structured interview consisting of four ‘sweeps’ 
(Crandall et al., 2006). In sweep one, interviewees are asked to recall a critical incident 
(defined in the previous section) and describe the sequence of events in which the incident 
occurred. For this study, interviewees were asked to recount what happens in a ‘routine’ 
mathematics class before they are asked to recall a critical incident with interviews taking 
about one hour. There are two reasons for this: teaching routines can vary from teacher to 
teacher, so having the interviewee describe their routines will provide the interviewer with 
a clearer picture of which aspects of the critical incident are outside of the teacher’s routine 
behaviours; and that this step will help the interviewee identify non-routine incidents. 

In sweep two, the information concerning the critical incident gathered in sweep one is 
reviewed and put into a timeline. For the purposes of this study, it is predicted that precise 
timeline information will not be required. Hence, sweep two will entail ensuring that 
events described in sweep one are in the correct sequence, with rough estimates of time 
between key events. In sweep two, decision points are identified. These are points at which 
a change in course occurred. Thus, while proceeding with a teaching plan may constitute a 
decision, the aim of sweep two is to identify points at which there was cause to reassess the 
course of action. 

In sweep three, decision points are investigated in more detail. Interviewees are asked 
to recount what they noticed about the situation (cues) around the decision point and what 
they were thinking at the time. In this study, teachers will be probed regarding their mental 
picture of the class and individual students, their understanding of the mathematics 
occurring at each point and their goals as the incident unfolds. 

The final sweep involves posing hypothetical questions about each decision point. In 
this study, hypothetical questions addressed hypothetical situations where the mathematical 
content was different (what if the student had been drawing a triangle instead of a square?), 
the teacher had been different (what if the teacher was in their first year of teaching in this 
situation?), the students had been different (what if the students hadn’t studied that last 
week?) and the content area had been different (what if this was a history lesson?). 

For the purposes of this current study, the data were coded as either pertaining to 
teachers’ cognition during the decision making incident or not. A more detailed analysis of 
this data set is underway and is envisaged as being reported in future publications when the 
pilot study has been completed. Particular note was taken of whether an alternative course 
of action generation was deliberative (concurrent weighing of costs and benefits - as in 
Schoenf eld’s DM model) or serial (think of options one at a time and mentally simulate 
their consequences until a workable solution is found - as in Klein’s RPD model). 
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For this pilot of CDM in education, participants were selected from primary teachers 
known to the researcher. CDM relies on capturing experts’ cognition. The expertise of 
participants was decided by the researcher - based on his experience working as a 
Numeracy Specialist in Victorian Government Schools who has engaged in coaching 
activities with many teachers. This is by no means intended to be a highly reliable measure 
of teacher expertise. It is envisaged that, if the pilot study suggests that CDM can be used 
to investigate expert teacher DM, subsequent studies, employing more rigorous 
assessments of teacher expertise, will be conducted. For the purposes of testing the 
viability of CDM, a sample of convenience, drawn from the researchers’ professional 
contacts, has been used. While this paper reports preliminary results from three CDM 
interviews, the pilot aims to collect data from five teachers. 

Results 

CDM interviews were able to generate data regarding the in-class DM of the three 
interviewees. Interviewees were able to recall critical incidents which related to students’ 
mathematical thinking and to provide details about each incident. Discrete cases were 
recalled with a reasonable amount of detail. Interviewee responses often took careful 
consideration. For example, none of the interviewees could initially explain how they came 
up with the alternative course of action used. After sweep three of the interview, 
interviewees were able to examine the information which was available to them before the 
decision and highlight which information had been at the forefront of their mind. CDM 
was able to facilitate interviewees’ ability to reflect on their cognition during DM. Of all of 
the data coded, 60% pertained to the interviewees’ cognition during the critical incidents 
investigated. 

The cognitive processes of all interviewees were remarkably similar and in the three 
cases reported here. The first alternative course of action generated was ‘reasonably good’ 
and was immediately enacted. Figure 1 has been created in order to visually represent the 
cognitive processes of the three interviewees. There are two agents in the figure: the 
classroom (which produces student cues in the form of body language and utterances) and 
the teacher (who enacts lesson plans, assesses the situation, decides that the course of 
action needs to change, then generates, assesses and enacts new courses of action). There 
was no evidence of concurrent deliberation between competing options. 

All three interviewees developed situation assessments - a mental picture of what is 
happening now - at various times leading up to and after the critical incident. Preliminary 
analyses of the environmental cues which were attended to, point to students’ body 
language and utterances (such as “huh?” and under-the-breath muttering, rather than actual 
sentences) as being attended to keenly by the interviewees. However, in order to read what 
students were thinking during the lesson, teachers used a process of self-monitoring. All 
interviewees described “listening to myself’ as they taught. This involved critical 
assessment of their own teaching performance from the perspective of a student. This kind 
of self-monitoring helped build situation assessments like, “I knew things were going 
badly. I was thinking that, if I was sitting on the floor listening to a grown up say these 
things, it wouldn’t make much sense”. This process of self-monitoring helped develop 
projections of the cause of student difficulty (“I thought that students were finding it [the 
teacher’s explanation] too rule-based so they couldn’t see the connection”) and helped 
generate ‘reasonably good’ alternative courses of action (“I needed a context that the kids 
would be familiar with, if they were going to see how it worked - then, bang! Temperature! 
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The kids will get this [subtraction with negative numbers], if I make it a story about 
temperature”). 

After the new course of action was adopted, student cues and the process of self- 
monitoring continued to determine whether the new course of action had the desired effect 
on student thinking. In all cases, interviewees took changes in students’ body language, 
utterances (“oh, I get it”), conversation between students and student mathematical activity 
to gauge the success of the new course. In all three cases, the new course of action seemed 
to work and, so, no new course of action was generated. 

Interviewees expressed that their goals during the lesson stayed the same. While the 
desire to develop a new course of action was expressed in terms of goals (“I wanted the 
kids to get it conceptually, and I could see that the way I was teaching wasn’t going to get 
that across”), the goals which drove the first and alternate courses of action were the same 
in the view of the interviewees. 



Figure 1: Proposed model of the process of teachers’ in-class DM 


Discussion 

The three primary mathematics teachers interviewed using CDM were able to recall 
incidents where they needed to make a challenging decision in real-time in order to 
influence students’ mathematical thinking. The fact that these kinds of critical incidents 
could be recalled in detail enabled the use of CDM as a means to capture their cognition 
during these incidents. This addresses the first research question of the pilot study and 
suggests that CDM may be able to be applied to the context of primary mathematics 
teaching. 

For the second research question of this study, the data provided by the 3 interviewees 
indicated there was one key decision point in each lesson where adaptive expertise was 
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required. The bulk of the data generated by CDM pertained to teachers’ cognition during an 
episode of adaptive expertise (60% of the total data). Hence, if one’s goal is to understand 
the cognitive processes at play in teachers’ in-the-moment DM, CDM may be able to 
generate useful data. Similarities between the three interviewees’ cognitive processes may 
have implications for current models of teachers’ DM in primary mathematics classrooms 
(research question 3). Figure 1 represents the common sequence of teacher thinking and 
observation described by interviewees. The process described is more similar to NDM 
accounts of DM, yet pre-existing NDM models (such as Klein’s RPD model) are not a 
perfect match. 

NDM studies from outside of teaching would suggest that cues in the situation generate 
plausible courses of action. Lipshitz et al. (2001) characterised this process as ‘situation- 
action matching decision rules’ and argued that experts do x because of situation y. If one 
only considers the information interviewees attended to which was in the classroom 
environment, the information was remarkably similar despite differences between 
situations. Students’ body language and vocal utterances were attended to the most, yet, at 
best, these cues were used to ascertain whether students understood what was happening, 
but could not be used to ascertain the nature of the misunderstanding. Slumped posture, 
‘confused eyes’ and utterances such as, “huh?” were attended to whether student confusion 
related to subtraction of negative integers in a Grade 5 class or subtracting one-digit 
numbers in a Prep class, but these cues alone did not suggest how to address the particular 
student misunderstanding relating to the specific mathematical content being taught. In 
order to build a situation assessment, there is some evidence that interviewees used a 
process of self-monitoring that involved critically attending to their own teaching from the 
perspective of a student. As one interviewee put it, “I was listening to myself and thinking, 
would this make sense? If I was one of the kids, would I get what’s going on here?” When 
the environmental cues signalled a lack of comprehension on the part of the students 
(shown on the left hand side of Figure 1), this process of self-monitoring (shown on the 
right-hand side of Figure 1) was used to develop a situation assessment which included 
possible causes of student confusion. Interviewees stated that, in order to make the 
assessment that an alternative course of action was required, indicators from either student 
cues or the process of self-monitoring were required. The assessment that no alternative 
was needed required both student cues and the process of self-monitoring to be going well. 

This pilot provides some evidence that expert teachers’ DM follows patterns similar to 
experts in other areas - experts generate few options of alternative action because the first 
options generated are ‘reasonably good’. NDM models postulate that options generated are 
assessed serially by mental simulation until a workable solution is found. While there is 
evidence that mental simulation was used to assess options, all three interviewees enacted 
the first option considered. Hence the serial generation of alternative options present in 
Figure 1 has not yet been observed in this pilot study. None of the interviewees suggested 
that the options they came up with were optimal, but all interviewees claimed that the 
alternative courses of actions they embarked on worked to some degree. This suggests that 
expert teachers do not engage in concurrent deliberation between options - weigh up the 
benefits and disadvantages of multiple courses of action until an optimal solution is found 
- in real-time decision making moments. In fact, there was no evidence of deliberation in 
any of the interviews. Thus, based on this small sample, if one’s goal is to describe the 
cognitive processes that expert teachers actually employ when making decisions in-class, a 
model that incorporates aspects of NDM seems promising. 
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Conclusion 

This pilot study aims to assess the viability of CDM in education and provide a source 
of data which could suggest descriptive models of the processes that experts follow in 
making a challenging decision in a real-time, classroom environment. The ultimate goal of 
the research project, of which this pilot study is part, is to develop methods for supporting 
the development of in-the-moment DM skills for primary maths teachers. As Cannon- 
Bowers and Bell (1997, p. 100) stated, “the most fruitful way to characterise NDM- 
consistent training might be to view it as a mechanism to support natural decision-making 
processes, and as a means to accelerate proficiency or the development of expertise”. This 
paper provides initial analysis of the data generated. Further analysis aims to test the 
tentative model of the natural decision-making processes of expert teachers proposed in 
Figure 1. More detailed coding of the pilot study’s data may be able to capture some of the 
knowledge, skills, environmental cues and mental schemata which support that DM 
process. Pre-existing NDM models, such as Klein’s RPD model, would direct attention to 
environmental cues as cues suggest courses of action. However, at least in the challenging 
cases of DM presented by interviewees, teacher DM was not entirely based on matching 
environmental cues to courses of action. Teachers’ situation assessments also relied on a 
process of self-monitoring, as student thinking is a less observable phenomenon than the 
kinds of phenomena engaged with in firefighting or warfare - areas where NDM models 
were developed. Lipshitz and Ben Shaul (1997) suggested adding a construct of mental 
schemata to Klein’s RPD model. The process of self-monitoring described by interviewees 
suggests that a construct such as mental schemata might play an important role in teacher 
DM which seeks to develop situation assessments of students’ mathematical thinking. 
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